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Conditional Independence 


1.4.4 Conditional Independence 


As we mentioned earlier, almost any concept that is defined for probability can also be extended to conditional 
probability. Remember that two events A and B are independent if 

P(A fl B) — P(A)P(B), or equivalently, P(A\B) = P(A). 


We can extend this concept to conditionally independent events. In particular, 

Definition 1.2 

Two events A and B are conditionally independent given an event C with P(C ) > Oif 


P(A n B\C) = P(A\C)P(B\C) 


Recall that from the definition of conditional probability, 


P(A\B) 


P(A n B) 
-P(-B) 


if P(B) > 


0. By conditioning on c, we obtain 


P(A\B,C) 


P(AHB\C ) 
P{B\C) 


if P(B\C), P{C ) 7 ^ 0. If A and B are conditionally independent given C , we obtain 


( 1 . 8 ) 


P(A\B,C) 


P{AnB\C) 

P(B\C) 

P(A\C)P(B\C) 

P(B\C) 

P(A\C). 


Thus, if A and B are conditionally independent given C , then 

P(A\B,C) = P{A\C) (1.9) 

Thus, Equations E 8 and E9 are equivalent statements of the definition of conditional independence. Now let's look at an 
example. 


Example 1.27 

A box contains two coins: a regular coin and one fake two-headed coin (P(H) = 1). I choose a coin at random and 
toss it twice. Define the following events. 

• A= First coin toss results in an H. 

• B= Second coin toss results in an H. 

• C= Coin 1 (regular) has been selected. 

Find ^(-AIC), P(B\C), P(A PI B\C), P(A), P(B), and P(A D E). Note that A and B are NOT 
independent, but they are conditionally independent given C. 
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• Solution 

o We have P(A\C) — P(B\C) = . Also, given that Coin 1 is selected, we have 

P(A n B\C) = \ = To find P(A),P(B), and P(A ft B) , we use the law of total 

probability: 

P(A) = P(A\C)P(C) + P(A\C C )P(C C ) 

= 1.1 + 1.1 

2 2 ' ± 2 

_ 1 
4 ' 

Similarly, P(B ) = -jj-. For P(A (~l B), we have 

P(A n B) = p(a n B\C)P(C) + P(A n b\c c )p(C c ) 

= P(A\C)P(B\C)P(C) + P(A\C c )P(B\C c )P(C c ) 

(by conditional independence of A and B ) 



As we see, P(An B) = ^ P(A)P(B) — -Jq , which means that A and B are not independent. We can also 

justify this intuitively. For example, if we know A has occurred (i.e., the first coin toss has resulted in heads), we would 
guess that it is more likely that we have chosen Coin 2 than Coin 1. This in turn increases the conditional probability that 
B occurs. This suggests that A and B are not independent. On the other hand, given C (Coin 1 is selected), A and B 
are independent. 

One important lesson here is that, generally speaking, conditional independence neither implies (nor is it implied by) 
independence. Thus, we can have two events that are conditionally independent but they are not unconditionally 
independent (such as A and B above). Also, we can have two events that are independent but not conditionally 
independent, given an event C. Flere is a simple example regarding this case. Consider rolling a die and let 

A = { 1 , 2 }, 

B -{2,4,6}, 

C — { 1 ) 4 }- 

Then, we have 

P(AriB) = i = PlAjPfB). 

Thus, A and B are independent. But we have 

P(A\C) = ^,P(B\C)= 1 --, 

P(AnB\C ) = P({2}\C) = 0. 
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Thus 

P(AnB\C) ± P(A\C)P{B\C), 

which means A and B are not conditionally independent given C . 

<— previous 
next —> 
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